Temporal Registration using a Kalman Filter for Augmented Reality Applications

Augmented Reality uses see-through head-mounted displays to superimpose synthetically generated information on a three-dimensional scene. Information is rendered in alignment with physical objects to enhance the user’s ability to perceive and interact with the world. A significant technical challenge related to Augmented Reality is determining the transformation that will correctly align the synthetic data with corresponding physical objects. This is made more difficult by the fact that the user and the scene may be in motion. Alignment must be both stable and accurate in order to produce the “illusion” that synthetic objects are an integral part of the environment. This paper presents a tracking algorithm capable of computing the three-dimensional pose of objects with respect to a headmounted camera as they are moved in the scene. Using the fixed transform from the headmounted camera and the see-through device, information can then be displayed in alignment with the current view of the object being tracked. In contrast to approaches that solve for the absolute position of the viewer within a fixed geometry [2,16], this object-centered approach to Augmented Reality models motion of both the object and the camerasdisplay system mounted on the user’s head using a single transformation involving six parameters. Objects to be augmented are first recognized as one of several stored within a database. Once recognized, appropriate feature points are extracted from the image for tracking and auxiliary information, associated with the object model is rendered into the current view. Accurate pose information is maintained using a Kalman filter that combines the predictions of a state-model and the pose of the object computed from tracked feature points and a three-dimensional pose recovery algorithm. The technique is demonstrated on a database of mechanical prototype parts, using a headmounted camera system. We explore the utility of the system in a manufacturing environment where an inspector requires access to CAD information related to a prototype part in order to facilitate qualitative decisions about the object. Results demonstrate that the algorithm is capable maintaining accurate temporal registration between the object and headmounted system at nearly 22Hz on a Sun Ultra Sparc-20 Workstation.